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Executive Summary 


Overall students in Missouri public schools continue to perform quite well. Of Missouri's 
523 districts, 319 districts received the Distinction in Performance recognition from the 
Department of Elementary and Secondary Education in 2008 for achievement on the 
Missouri Assessment Program exam. 


Through a closer examination in the variations in student achievement in Missouri 
schools, two primary findings emerged. First, consistent with decades of research, 
socioeconomic status has a strong negative correlation with student achievement. 
Second, in high-poverty, high-achieving districts, the clearest distinction between those 
schools and high-poverty, low-achieving schools is in the teachers’ reporting of the 
quality of the learning environment in the schools. 
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SECTION 1 


Highlights and Supplementary Information 
to the Missouri Public School Accountability Report (December 2008)' 


Each year the Department of Elementary and Secondary Education (DESE) produces 
the Missouri Public School Accountability Report. Information contained in Section 1 is a 
companion piece to that report. 


District Accreditation 2008 

Accreditation is determined by review through the Missouri School Improvement 
Program (MSIP). Each year 20% of Missouri school districts participate in the MSIP 
review so that each MSIP cycle is a five-year process. 


Charter Schools 

Charter schools refers to the number of individual charters that have been approved by 
the State Board of Education. The number of school buildings is referred to as the 
number of campuses. (Table 1.) 


Table 1. Missouri Charter Schools 














# of schools # of campuses 
Kansas City 18 24 
St. Louis 10 19 














Free or Reduced Price Lunch 
Free or reduced price lunch is based on a national standard of income eligibility? and is 
the most commonly used proxy for socioeconomic status in school data. 


Missouri Graduation Rate 


Table 2. Missouri Graduation Rates 





2004 2005 2006 2007 2008 




















Graduation rate 85.6 86.0 85.8 86.3 85.2 














' All data come from the Department of Elementary and Secondary Education unless otherwise noted. 


2 

To qualify for free lunch, family income must be at or below 130% of poverty. Family incomes between 130% and 185% of poverty qualify for 
reduced-price lunch. In FY09, 130% of poverty for a family of four was $27,560. 185% of poverty for a family of four was $39,220. Source: 
National School Lunch Program, http://www.fns.usda.gov/cnd/Lunch/AboutLunch/NSLPFactSheet.pdf. 
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Graduation Rates vs. Dropout Rates® 
Graduation rate refers to the percentage of students who graduated out of their cohort 


entering in the 9"" grade. 
[N of grads / (9'"-12" grade cohort dropouts + grads)] x 100 


Dropout rate refers to the percentage of students who leave school in a given year as a 
percentage of the average enrollment. 


[N of dropouts / (total September enrollment + net transfers in) / 2] 


Completion of Postsecondary Education* 
These data represent the percentage of the population from age 25 to 65 holding a 


degree as of 2007. 


Missouri AA/AS/AAS or higher — 7% Missouri BA/BS or higher — 28% 
National AA/AS/AAS or higher — 8% National BA/BS or higher — 29% 


National Assessment of Educational Progress (NAEP) 


NAEP is not required of all Missouri districts. The numbers reported in the Missouri 
Public School Accountability Report reflects a total of 12,200 students in 155 districts. 
The sample was selected by stratified random sampling based on locale and minority 
enrollment. The National Center of Education Statistics (NCES) selects a sample of 
students and schools representing each state. 


Adequate Yearly Progress (AYP) 
AYP is the performance requirement mandated by No Child Left Behind (NCLB). AYP 


may be achieved by raw score averages that meet the required levels or by meeting the 
goals of an estimated growth trajectory that would have students at the required levels 
within the timeframe required by NCLB. 


Definition of Highly Qualified Teachers 

NCLB definition of highly qualified teacher. To be deemed highly qualified, teachers 
must have: 1) a bachelor's degree, 2) full state certification or licensure, and 3) proof 
that they know each subject they teach.° 


> There are different calculation methods for graduation rate and dropout rate. The definition listed is the one used 
by DESE. The formulas can be found at 
http://www.dese.mo.gov/divspeced/DataCoord/PDF/CalcMethodGradDO_US_vs.pdf. 

* Source: American Community Survey. U.S. Census Bureau. http://www.census.gov/acs/www. 

° Source: U.S. Department of Education. New No Child Left Behind Flexibility: Highly Qualified Teachers. 
http://www.ed.gov/nclb/methods/teachers/hqtflexibility.html. 


Student-Teacher Ratios 2004-2008 
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Table 3. Student to Teacher Ratios 2004-2008 














2004 2005 2006 2007 2008 
Students per teacher 14 14 13 13 13 
Students per 19 19 18 18 18 
classroom teacher 




















The MSIP minimum quality standard for student-teacher ratios is listed below. The 
desirable standard is noted parenthetically. 


Teachers’ Years of Experience and Education Level 


Table 4. Teacher Credentials 






























































2004 2005 2006 2007 2008 

Average years of 12.9 12.8 12.6 12.6 12.4 
experience 
Percentage with 47.5% 49.6% 49.9% 50.6% 51.3% 
masters degree or 
higher 
ACT and SAT scores 2007-08 
Table 5. ACT Means and Percentage Tested 

Missouri National 
ACT composite 21.6 21.1 
Percent of graduates tested 69% 43% 
Table 6. SAT Means and Percentage Tested® 

Missouri National 
SAT critical reading 594 502 
SAT mathematics 597 515 
SAT writing 584 494 
Percent of graduates tested 5.6% 45.4% 




















° Source: The College Board SAT. http://professionals.collegeboard.com/profdownload/Missouri_CBS_08.pdf. 
National percentage tested was calculated from the College Board number tested divided by the total number of 
graduates as reported by ACT to keep the denominator consistent between ACT and SAT test takers. 
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Discipline Incidents 

Below is a breakdown of discipline incidents by offense. Examples of offenses that 
would be included in Other are any offenses which do not fit into any of the other 
categories but result in a student being removed from the classroom for one half day or 
more. In 2006, the rule regarding the category of Ofher changed from a student being 
removed from the classroom for more than 10 days to a student being removed from 
the classroom for one half day or more. As a result, the number of Other discipline 
incidents reported in 2006 sharply increased. 


Discipline Incidents 2004-2008 
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SECTION 2 


An Overview of Variables Related to Academic Achievement and Growth 


Input Variables 
Schools vary in the resources they have to educate their students. Some resources 


such as per pupil expenditure are a reflection of the property values and tax rate of a 
community. Other resource variables such as teachers’ credentials can be affected to a 
limited degree by the success a school has in recruiting and retaining teachers with 
certain credentials. Additionally, students vary in the prior achievement, skill set, and 
environmental influences they bring with them into the classroom. Student-level input 
variables such as socioeconomic status have been shown through numerous studies to 
have a strong relationship with a student’s academic achievement. 


Process Variables 

The impact of the processes that occur between the inputs and outcomes is the 
foundation of school effectiveness research.’ Knowing which processes have the 
strongest correlation with student achievement status and growth is the basis for 
Section 3 of this report. 


As part of the MSIP review process, DESE administers the Advanced Questionnaire 
(AQ) to faculty, parents, and students. The AQ was developed and is revised each 
MSIP cycle by a team of DESE staff, Office of Social and Economic Data Analysis 
(OSEDA) staff, and practicing educators. The AQ was developed based on current 
research in school effectiveness and best practices, and it includes questions about the 
learning environment of the school. Means for individual questions are grouped 
thematically, and scale variables are created for the different indicators. ° 


For example, the scale variable Schoo! Leadership includes the following individual 
questions. Items include responses on a scale of 1 (strongly disagree) to 5 (strongly 
agree): 


e Instructional time available to teachers is protected from all types of interruptions. 

e In our school teachers are encouraged to be instructional leaders. 

e My school’s principal fosters shared beliefs and a sense of community and 
cooperation. 

e My school’s principal monitors the effectiveness of school practices and their 
impact on student learning. 

e Our principal identifies issues in the school that could potentially become 
problems. 





"For example, see Robert Marzano, What Works in Schools. 

Each scale is tested for reliability using Cronbach’s alpha. This coefficient, ranging from 0 to 1| indicates the 
degree to which the individual items collectively measure what they were intended to measure. In social science 
research, the Cronbach’s alpha should be greater than 0.7 to be considered reliable. The Cronbach’s alpha for the 
School Leadership scale was 0.91. 


Joint Committee on Education — Study of the Public Schools of Missouri 


e My school’s principal systematically engages faculty and staff in discussions 
about current research on teaching and learning. 

e Our principal promotes innovation. 

e The mission of the school is clearly defined. 

e There are open channels of communication among students, staff, and 
administrators. 


For this section of the study, the following scale variables were examined for their 
relationship to growth in student achievement.? Noted parenthetically is the group 
completing a questionnaire on that particular variable. 


School Leadership — Identifies the degree to which leadership is perceived as 
effective in improving student learning. (faculty) 

Parental Involvement — Identifies the degree to which parents are viewed as 
partners in the education of their children. (faculty) 

Safe and Orderly Environment — Identifies the degree to which the school 
environment is safe and orderly. (faculty, student, parent) 

School Climate - Identifies the degree to which all students feel respected and 
valued. (faculty, parent, student) 

Guaranteed and Viable Curriculum - Identifies the degree to which essential 
curriculum has been identified in the district and the degree to which 
students have adequate opportunity to learn the content. (faculty) 

Professional Development - identifies the impact of professional development on 
improving learning for all students. (faculty) 

Community Capital - identifies the level of commitment and support by the 
community for the school. (faculty) 

Efficacy and expectations - identifies the degree to which teachers and students 
believe that they are capable of impacting student achievement. (students, 
parents) 

Classroom management - identifies the degree to which educational personnel 
establish and enforce classroom management processes that enhance 
student learning. (students) 


Outcomes 

Two measures of achievement are included as part of this study. The first measure of 
student achievement is the median growth percentile for the school between the 
academic years 2006-07 and 2007-08. Growth measurements attempt to isolate the 
effect of the school/teacher on student achievement. The second measure of student 
achievement is percentage of students in the school scoring at the level of proficient or 
advanced on the MAP. 





* The composite variable descriptions were provided by the Office of Social and Economic Data Analysis (OSEDA). 
OSEDA provides assistance to DESE with the collection and analysis of MSIP data. 
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SECTION 3 


An Examination of Variables Related to Academic Growth and Achievement in 
Missouri Elementary and Middle School Students 


A sample of elementary and middle schools (n = 308) reflecting the demographic profile 
of Missouri were selected for this study.'° Each school had its 4"" cycle MSIP review 
during the 2007-08 academic year. 


Figure 1. Socioeconomic Status and Mathematics Achievement 





As was noted in Section 2, 
al socioeconomic status has 
been shown to have a strong 
relationship with student 
achievement. Figures 1 and 
2 illustrate the strong, 
negative correlation between 
MAP achievement and 
socioeconomic status for the 
sample of schools in this 
study. 


Mathematics percent proficient or advanced 2008 














Correlation coefficients 
Percent Free or Reduced Lunch range from +1 O (perfect 


positive correlation) to -1.0 
(perfect negative correlation) 


Figure 2. Socioeconomic Status and Communication Arts Achievement 





a Mathematics 
° = - 0.802 


Communication arts 
r=-0.791 


Correlations are statistically significant 
at the 0.01 level (2-tailed). 











Communication arts percent proficient or advanced 2008 





Percent free or reduced lunch 


'° OSEDA verified the representativeness of the sample through the following method: “The results indicated that 
the SAMPLE was statistically significantly different from ALL SCHOOLS, with an alpha of 0.05, for both 
PERCENT MINORITY and LOCALE TYPE. In order to create a representative sample, schools with 25% or more 
minority students and which were (‘Large City or Mid-size City’ or 'Large Town or small town') were randomly 
selected for removal until the chi-square analysis no longer showed statistically significant results.” 
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The previous illustration is not intended to convey a lack of ability for schools to affect 
student achievement, but rather to acknowledge the influence of socioeconomic status 
so that variables may be examined independent of this influence. 


In Figures 3 and 4, growth in student achievement is plotted on the horizontal axis and 
percent of students proficient or advanced is plotted on the vertical axis. In addition, 
schools have been grouped into three categories based on their percentage of students 
eligible for free or reduced lunch. 


Figure 3. MAP Mathematics 


Percent free or reduced 
lunch 


© Less Than 25% 
O 25% to 49% 
© 50% or More 


Sustaining Excelling 


Mathematics percent proficient or advanced 2008 


Improving 
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Communication arts percent proficient or advanced 2008 
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Figure 4. MAP Communication Arts 


Sustaining Excelling Percent ihe reduced 
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Using communication arts achievement data, resource variables are compared for 
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schools in the quadrants of Excelling, Improving, and Underperforming. The previous 
graphs reinforce the influence of socioeconomic status on achievement. In order to 
draw comparisons independent of socioeconomic status, only those schools with a 
population of 50% or higher free or reduced lunch eligibility are included. 


Table 7. Comparison of Resource Data Across Achievement Subgroups 
















































































EXCELLING IMPROVING UNDERPERFORMING 
Proficiency and Growth | Proficiency less than Proficiency and Growth | All schools 
greater than 50% and 50%; Growth greater less than 50% and in sample 
FRL > 50% (n=11) than 50% and FRL > 50% (n=89) (n=308) 
FRL > 50% (n=48) 
Resource data 
Teachers with regular 98.5 97.3 97.2 97.8 
certification (%) 
Highly qualified 96.4 96.7 96.8 97.2 
teachers (%) 
Teachers with 38.4 46.1 42.5 48.7 
master’s degree (%) 
Teachers average 11.4 12.8 13.2 12.5 
years of experience 
Average teacher’s $36,442 $42,391 $43,787 $43,520 
salary 
Student attendance 94.8 94.3 92.8 94.5 
rate 
Student-classroom 16.5 15.4 16.8 16.1 
teacher ratio 
Per pupil expenditure $7,953 $9,152 $9,290 $8,484 
(median) 
Table 8. Discipline Incident Rate by Achievement Subgroup" 
EXCELLING IMPROVING UNDERPERFORMING 
Proficiency and Growth | Proficiency less than Proficiency and Growth | All schools 
greater than 50% and 50%; Growth greater less than 50% and in sample 
FRL > 50% (n=11) than 50% and FRL > 50% (n=89) (n=308) 
FRL > 50% (n=48) 
Discipline Incidents 
(rate per 100 students) 
Alcohol .045 .000 .018 023 
Drug .255 .081 .067 .092 
Violence .000 .079 .034 .052 
Weapon .009 133 .184 .095 
Tobacco .000 .000 011 .004 
Out-of-school 582 1.283 2.426 1.261 
suspension 
Expulsion .000 012 .000 .004 




















'' A graphic representation of these data has been appended to this report. See page 17. 
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Observations 


The percentage of teachers with regular certification was slightly higher in the 
Excelling subgroup than in other subgroups as well as higher than the average of 
the full sample. 

The percentage of highly qualified teachers was roughly equivalent across all 
three subgroups, but slightly lower than the average of the full sample. 

The percentage of teachers with a master’s degree or higher is lower in the 
Excelling group. However, this should be interpreted with caution due the small 
number of schools in the sample and the locale of those schools which may 
make continuing education less accessible. Worth noting is that all of the three 
higher poverty school subgroups have a lower percentage of teachers with 
master’s degrees than in the sample as a whole. 

The attendance rate for the Excelling and Improving subgroups was roughly 
equivalent to the attendance rate of the full sample and slightly higher than the 
attendance rate of the Underperforming subgroup. 

The student to classroom teacher ration was roughly equivalent across the full 
sample, the Excelling subgroup, and the Underperforming subgroup. Student to 
classroom teacher ratios were lowest in the Improving subgroup. 

The per pupil expenditure is included but is not a valid measure in this context. 
Per pupil expenditures are a reflection of the property values and the tax rate of 
the school district. In the Excelling subgroup, the majority of schools are rural 
schools with a lower tax base. In addition, factors such Title 1 federal funding and 
extra weighting in the foundation formula for some populations of students (e.g., 
English language learners) affect the bottom line on a district’s per pupil 
expenditure. 

Similar to the per pupil expenditure, the difference in teacher salary can be 
attributed in part to locale of the schools in terms of tax base, cost of living, etc. 
The discipline incident rate shows the most distinction between achievement 
subgroups in the rate of out-of-school suspensions (OSS). The Underperforming 
group has twice the OSS rate of the Improving group and more than four times 
greater an OSS rate than the Excelling schools. 
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On the AQ, respondents rated a series of individual items related to the learning 


environment of their school from 1 (strongly disagree) to 5 (strongly agree). Mean 


scores are calculated for each responding group for each school. Those school means 
within each category are then averaged to determine a mean for the subgroup. 


Table 9. Comparison of Process Data Across Achievement Subgroup 

































































EXCELLING IMPROVING UNDERPERFORMING _ | All schools in 

Proficiency and Proficiency less Proficiency and Growth sample 

Growth greater than 50%; Growth less than 50% and FRL_ | (n=308) 

than 50% and FRL | greater than 50% > 50% 

> 50% and FRL > 50% (n=89) 

(n=11) (n=48) 
Process data (AQ scale 1-5) 
FAC — School leadership 4.4 4.2 4.0 4.1 
FAC — Parental involvement 4.4 4.2 4.1 4.2 
FAC — Safe, orderly environment 4.3 4.0 3.8 4.0 
FAC — School climate 4.4 4.3 4.1 4.3 
FAC — Guaranteed, viable 4.2 4.0 3.9 4.0 
curriculum 
FAC — Professional development 4.2 4.1 3.9 4.0 
FAC — Community capital 4.3 3.9 ow 4.0 
STU — Classroom management 3.9 3.7 3.7 3.8 
STU — Efficacy, expectations 4.5 4.5 4.5 4.5 
STU — Equity 4.0 4.1 4.0 4.1 
STU — Feel safe 4.1 4.1 4.0 4.1 
STU — Instructional strategies 3.2 3.2 3.3 3.2 
STU — School climate 4.0 4.1 4.0 4.1 
PAR — Efficacy, expectations 4.0 3.9 4.0 4.0 
PAR — Safe, orderly environment 4.0 3.8 3.7 3.9 
PAR — School climate 3.8 3.7 3.8 3.8 

















Observations 


e Across the three populations included in the survey, the differences among the 
responses of students and parents in Excelling, Improving, and Underachieving 
schools is negligible. '? In particular, the students’ sense of efficacy is exactly the 
same across all three subgroups. Students in these higher poverty schools 
shared an equal sense of their own capacity to be successful academically. 


'? The use of the mean of group means is typically not recommended in statistical analysis because the weighting 
element is lost. However, in this situation, the intent was to give equal weight to each school in the subgroup 
considering the school mean as a reflection of the overall building culture and so as not to bias the subgroup mean in 
favor of schools with a greater number of faculty. Subgroup means were calculated at the teacher level and are 
available upon request. 

' The results of the parent AQ should be interpreted with caution. The required response rate for parents is 25% 
making it much easier to have biased results. 
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The clearest differences across subgroups are seen in the faculty responses. 
Factors which describe a positive learning environment are higher across every 
scale variable for teachers in Excelling schools. 

The AQ scales where there is the most difference in student responses is related 
to teacher practices (classroom management). 





Mean scale score 


Figure 5. Faculty Perspectives on the Learning Environment 
in schools with >50% FRL 
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Figure 6. Student Perspectives on the Learning Environment 
in schools with >50% FRL 
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Conclusion 

While the influence of socioeconomic status on student achievement is clearly seen in 
the data, it is worth noting the distinctions between practices within those schools that 
are excelling in spite of socioeconomic status and those schools that are 
underperforming. 


Survey data show that in high-poverty, high-achieving schools, teachers report a 
qualitatively different learning environment than in high-poverty, low-achieving schools 
reflected in their responses regarding school leadership, parent and community 
involvement, safety, curriculum alignment, and professional development. 
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Appendix 


Discipline Incidents by Achievement Subgroups 
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